Proposed methods for reviewing the outcomes of health research: the impact of funding by the UK's Arthritis Research Campaign

Background: External and internal factors are increasingly encouraging research funding bodies to demonstrate the outcomes of their research. Traditional methods of assessing research are still important, but can be merged into broader multi-dimensional categorisations of research benefits. The onus has hitherto been on public sector funding bodies, but in the UK the role of medical charities in funding research is particularly important and the Arthritis Research Campaign, the leading medical charity in its field in the UK, commissioned a study to identify the outcomes from research that it funds. This article describes the methods to be used. Methods: A case study approach will enable narratives to be told, illuminating how research funded in the early 1990s was (or was not) translated into practice. Each study will be organised using a common structure, which, with careful selection of cases, should enable cross-case analysis to illustrate the strengths of different modes and categories of research. Three main interdependent methods will be used: documentary and literature review; semi-structured interviews; and bibliometric analysis. The evaluative framework for organising the studies was previously used for assessing the benefits from health services research. Here, it has been specifically amended for a medical charity that funds a wide range of research and is concerned to develop the careers of researchers. It was further refined in three pilot studies. The framework has two main elements. First, a multi-dimensional categorisation of benefits going from the knowledge produced in peer reviewed journal articles through to the health and potential economic gain. The second element is a logic model, which, with various stages, should provide a way of organising the studies. The stock of knowledge is important: much research, especially basic, will feed into it and influence further research rather than directly lead to health gains. The cross-case analysis will look for factors associated with outcomes. Conclusions: The pilots confirmed the applicability of the methods for a full study which should assist the Arthritis Research Campaign to demonstrate the outcomes from its funding, and provide it with evidence to inform its own policies

Heuristics, not plumage: A response to Osterloh and Frey's discussion paper on ‘Borrowed plumes’

At its simplest, ‘How to avoid borrowed plumes in academia’ by Margit Osterloh and Bruno Frey (Osterloh & Frey, 2020) is a paper about how using Journal Impact Factor (JIF) to judge academic papers and their authors is a bad idea; their suggestion of why people nevertheless persist in doing so; and what might be done to stop them. More broadly, it is a paper about heuristics (or rules-of-thumb) – why they are used; when they should not be used; and how to stop them being used in those contexts. I agree with Osterloh and Frey that JIF is a bad heuristic for judging research, but I find their arguments about why it is used and what might be done to stop people using it unconvincing and impractical. In this short Note, I argue that the use of heuristics is inevitable and, if effectively selected, they can improve decision-making. The challenge for an individual is to decide which heuristics are worth using. The policy challenge is to dissuade people from using inappropriate heuristics – and doing this requires good evidence on how and why the heuristics are being used, something that is missing from Osterloh and Frey’s paper.Non

Project Retrosight. Understanding the returns from cardiovascular and stroke research: Case Studies

Copyright @ 2011 RAND Europe. All rights reserved. The full text article is available via the link below.This project explores the impacts arising from cardiovascular and stroke research funded 15-20 years ago and attempts to draw out aspects of the research, researcher or environment that are associated with high or low impact. The project is a case study-based review of 29 cardiovascular and stroke research grants, funded in Australia, Canada and UK between 1989 and 1993. The case studies focused on the individual grants but considered the development of the investigators and ideas involved in the research projects from initiation to the present day. Grants were selected through a stratified random selection approach that aimed to include both high- and low-impact grants. The key messages are as follows: 1) The cases reveal that a large and diverse range of impacts arose from the 29 grants studied. 2) There are variations between the impacts derived from basic biomedical and clinical research. 3) There is no correlation between knowledge production and wider impacts 4) The majority of economic impacts identified come from a minority of projects. 5) We identified factors that appear to be associated with high and low impact. This report presents the key observations of the study and an overview of the methods involved. It has been written for funders of biomedical and health research and health services, health researchers, and policy makers in those fields. It will also be of interest to those involved in research and impact evaluation.This study was initiated with internal funding from RAND Europe and HERG, with continuing funding from the UK National Institute for Health Research, the Canadian Institutes of Health Research, the Heart and Stroke Foundation of Canada and the National Heart Foundation of Australia. The UK Stroke Association and the British Heart Foundation provided support in kind through access to their archives

Project Retrosight. Understanding the returns from cardiovascular and stroke research: Methodology Report

Copyright @ 2011 RAND Europe. All rights reserved. The full text article is available via the link below.This project explores the impacts arising from cardiovascular and stroke research funded 15-20 years ago and attempts to draw out aspects of the research, researcher or environment that are associated with high or low impact. The project is a case study-based review of 29 cardiovascular and stroke research grants, funded in Australia, Canada and UK between 1989 and 1993. The case studies focused on the individual grants but considered the development of the investigators and ideas involved in the research projects from initiation to the present day. Grants were selected through a stratified random selection approach that aimed to include both high- and low-impact grants. The key messages are as follows: 1) The cases reveal that a large and diverse range of impacts arose from the 29 grants studied. 2) There are variations between the impacts derived from basic biomedical and clinical research. 3) There is no correlation between knowledge production and wider impacts 4) The majority of economic impacts identified come from a minority of projects. 5) We identified factors that appear to be associated with high and low impact. This report presents the key observations of the study and an overview of the methods involved. It has been written for funders of biomedical and health research and health services, health researchers, and policy makers in those fields. It will also be of interest to those involved in research and impact evaluation.This study was initiated with internal funding from RAND Europe and HERG, with continuing funding from the UK National Institute for Health Research, the Canadian Institutes of Health Research, the Heart and Stroke Foundation of Canada and the National Heart Foundation of Australia. The UK Stroke Association and the British Heart Foundation provided support in kind through access to their archives

Circular 52

This is the sixth publication in this format on grain performance trials in the Tanana River Valley. The first, published 5 years ago, included the results o f spring cereal-variety tests conducted at Fairbanks and Delta Junction during the 1978 and 1979 growing seasons. The variety-test results from the 1980, 1981, 1982, and 1983 growing seasons were annual publications. Included in this report are a weather summary, the 1984 variety-test results, and a plant-disease section.Introduction -- Standard Bushel Weights and Conversion from English to Metric Units -- Part I: Climatic Data for and Germplasm Evaluation: Tanana Valley Weather Summary: Table 1: Climatic Data for Delta Junction during the 1984 Growing Season, Table 2: Climatic Data for Fairbanks during the 1984 Growing Season; Barley Performance Trials: Table 3: Long-Term Average and Range of Yields for Barley Standard Varieties Grown at Fairbanks and Delta Junction, 1971-1984, Table 4: Barley Variety Trials Conducted at Delta Junction and Fairbanks during the 1984 Growing Season, Variety Descriptions, Table 5: Barley Varieties Tested at Fairbanks and Delta Junction, 1971-1984 -- Oat Performance Trials: Table 6 : Long-Term Average and Range in Yields for Oat Standard Varieties Grow n at Fairbanks and Delta Junction, 1971-1984, Table 7: Oat Variety Trials Conducted at Delta Junction and Fairbanks during the 1984 Growing Season, Variety Descriptions, Table 8 : Oat Varieties Tested at Fairbanks and Delta Junction, 1971-1984 -- Spring Wheat Performance Trials: Table 9: Long-Term Average and Range in Yields for Wheat Standard Varieties Grown at Fairbanks and Delta Junction, 1971-1984, Table 10: W heat Variety Trials Conducted at Fairbanks and Delta Junction during the 1984 Growing Season -- Variety Descriptions: Table 11: W heat Varieties Tested at Fairbanks and Delta Junction, 1971-1984 -- Part II: Plant-Disease Evaluation: Barley Diseases: Table 12: Summary of Diseases Observed on Barley Varieties under Field Conditions in the Delta-Clearwater Area, Snow M old Disease Com plex on W inter W heat and Lawn G ra s s e s, Diseases on O ther C r o p, Diseases O bserved on Crops during the 1984 Growing Season and T heir S ym ptom

Project Retrosight. Understanding the returns from cardiovascular and stroke research: Policy Report

Copyright @ 2011 RAND Europe. All rights reserved. The full text article and the summary of the article are both available via the links below.This project explores the impacts arising from cardiovascular and stroke research funded 15-20 years ago and attempts to draw out aspects of the research, researcher or environment that are associated with high or low impact. The project is a case study-based review of 29 cardiovascular and stroke research grants, funded in Australia, Canada and UK between 1989 and 1993. The case studies focused on the individual grants but considered the development of the investigators and ideas involved in the research projects from initiation to the present day. Grants were selected through a stratified random selection approach that aimed to include both high- and low-impact grants. The key messages are as follows: 1) The cases reveal that a large and diverse range of impacts arose from the 29 grants studied. 2) There are variations between the impacts derived from basic biomedical and clinical research. 3) There is no correlation between knowledge production and wider impacts 4) The majority of economic impacts identified come from a minority of projects. 5) We identified factors that appear to be associated with high and low impact. This report presents the key observations of the study and an overview of the methods involved. It has been written for funders of biomedical and health research and health services, health researchers, and policy makers in those fields. It will also be of interest to those involved in research and impact evaluation.This study was initiated with internal funding from RAND Europe and HERG, with continuing funding from the UK National Institute for Health Research, the Canadian Institutes of Health Research, the Heart and Stroke Foundation of Canada and the National Heart Foundation of Australia. The UK Stroke Association and the British Heart Foundation provided support in kind through access to their archives

A Proposal for an Environmental Decision Support System at the Regional Level: Concepts, Support Methodology, Tools and their Terminology

One of the goals of IIASA's research activities in the area of environmental quality modeling is the integration of data and models in a unified framework to assist decision makers with the management of complex environmental systems. Building on IIASA's work undertaken within the WELMM (Water, Energy, Land, Materials and Manpower) project of the former Resources and Environment Area and the work on Decision Support Systems of the former Management and Technology Area, a conceptual framework for an environmental decision support system (EDSS) has been developed and is presented in this paper. The proposed EDSS has been developed with the interest and the financial support of the CSI, the Center for Information Systems of the Regional Government of Piemonte, Italy. The main issue addressed by this paper is to devise a system assisting decision makers in tackling environmental problems at the regional level. These decisions are typically characterized by a combination of both structured (formalizable, described in a quantitative model) and unstructured elements (incomplete information, undefined cause-effect relationships, influence of political objectives, public perception, consideration of aesthetics, etc.). The proposed EDSS enables the user to use models and data, of relevance to a particular task, which are embedded in the EDSS in the form of a process information system. The specific feature of this process information system is that it contains processes of anthropogenic nature (the socio-economic activities being the cause of environmental impacts like power plants, industrial production units, etc.) as well as natural processes determining the spatial/temporal distribution and the extent of environmental quality changes (like the dispersion and deposition of air pollutants and their effect on human population, vegetation and wildlife). The system ensures that the data and models, which have been developed in the context of specific EDSS applications are documented right from the outset and become thus equally available for further use. This becomes especially important in view of the long-term effort to be put into the development of data and models dealing with the large number of environmental problems that governments, industry and academic institutions are confronted with at the regional level

Economies of scale and scope in publicly funded biomedical and health research: evidence from the literature

BACKGROUND: Publicly funded biomedical and health research is expected to achieve the best return possible for taxpayers and for society generally. It is therefore important to know whether such research is more productive if concentrated into a small number of ‘research groups’ or dispersed across many. METHODS: We undertook a systematic rapid evidence assessment focused on the research question: do economies of scale and scope exist in biomedical and health research? In other words, is that research more productive per unit of cost if more of it, or a wider variety of it, is done in one location? We reviewed English language literature without date restriction to the end of 2014. To help us to classify and understand that literature, we first undertook a review of econometric literature discussing models for analysing economies of scale and/or scope in research generally (not limited to biomedical and health research). RESULTS: We found a large and disparate literature. We reviewed 60 empirical studies of (dis-)economies of scale and/or scope in biomedical and health research, or in categories of research including or overlapping with biomedical and health research. This literature is varied in methods and findings. At the level of universities or research institutes, studies more often point to positive economies of scale than to diseconomies of scale or constant returns to scale in biomedical and health research. However, all three findings exist in the literature, along with inverse U-shaped relationships. At the level of individual research units, laboratories or projects, the numbers of studies are smaller and evidence is mixed. Concerning economies of scope, the literature more often suggests positive economies of scope than diseconomies, but the picture is again mixed. The effect of varying the scope of activities by a research group was less often reported than the effect of scale and the results were more mixed. CONCLUSIONS: The absence of predominant findings for or against the existence of economies of scale or scope implies a continuing need for case by case decisions when distributing research funding, rather than a general policy either to concentrate funding in a few centres or to disperse it across many

Data-Driven Abstraction-Based Control Synthesis

This paper studies formal synthesis of controllers for continuous-spacesystems with unknown dynamics to satisfy requirements expressed as lineartemporal logic formulas. Formal abstraction-based synthesis schemes rely on aprecise mathematical model of the system to build a finite abstract model,which is then used to design a controller. The abstraction-based schemes arenot applicable when the dynamics of the system are unknown. We propose adata-driven approach that computes the growth bound of the system using afinite number of trajectories. The growth bound together with the sampledtrajectories are then used to construct the abstraction and synthesise acontroller. Our approach casts the computation of the growth bound as a robust convexoptimisation program (RCP). Since the unknown dynamics appear in theoptimisation, we formulate a scenario convex program (SCP) corresponding to theRCP using a finite number of sampled trajectories. We establish a samplecomplexity result that gives a lower bound for the number of sampledtrajectories to guarantee the correctness of the growth bound computed from theSCP with a given confidence. We also provide a sample complexity result for thesatisfaction of the specification on the system in closed loop with thedesigned controller for a given confidence. Our results are founded onestimating a bound on the Lipschitz constant of the system and provideguarantees on satisfaction of both finite and infinite-horizon specifications.We show that our data-driven approach can be readily used as a model-freeabstraction refinement scheme by modifying the formulation of the growth boundand providing similar sample complexity results. The performance of ourapproach is shown on three case studies.<br

Designing out terrorism: human factors issues in airport baggage inspection

All air passenger baggage is screened at airports by means of 2-D X-ray imaging which results in a computer display of each luggage item that is then visually searched by an operator (screener) for the presence of potential threat items (e.g. knifes, guns, improvised explosive devices [IED]). Despite improvements in screener training and available technology (e.g. image enhancement functions, threat image projection, 3-D X-ray imaging) the performance of screeners is variable which leads to the potential for terrorist threat to aircraft and passengers. A new training scheme to improve performance in baggage screening is under development (EPAULETS: Enhanced Perceptual Anti-terrorism Universal Luggage Examination Training System) and some of the initial human factors issues that underlie variable screener performance are considered